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Abstract 

Let {{Zi, Wi) : i = 1, . . . , n} be uniformly distributed in [0, 1]"^ x 
G{k,d), where G{k,d) denotes the space of /c-dimensional hnear sub- 
spaces of W^. For a differentiable function / : [0, l]'^ — > [0, l]'^, we say 
that / interpolates {z,w) G [0, l]*^ x G{k,d) if there exists x G [0, l]'^ 
such that /(x) = z and f{x) = w, where /(x) denotes the tangent 
space at x defined by /. For a smoothness class of Holder type, 
we obtain probability bounds on the maximum number of points a 
function / G ^ interpolates. 



1 Introduction 

This paper is motivated by experiments in the field of Psychophysics |3] that 
study the ability of the Human Visual System at detecting curvilinear fea- 
tures in background clutter. In these experiments, human subjects are shown 
an image consisting of oriented small segments of same length dispersed in a 
square, such as in Figure [H 

The locations and orientations of these segments are either purely ran- 
dom (panel (a)) or a curve is actually "hidden" among purely random clutter, 

-'^This work was partially supported by NSF grant DMS-0603890. The author was at 
the Mathematical Sciences Research Institute while preparing the first draft. 

^The author would like to thank Emmanuel Candes, David Donoho, Bruce Driver, 
Bo'az Klartag and Allen Knutson for helpful discussions and references. 

^AMS 2000 subject classifications: Primary 60D05; secondary 62G10. 

'^Keywords and phrases: Grassmann Manifold, Haar Measure, Pattern Recognition, 
Kolmogorov Entropy. 



1 



(a) Under Hq 
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(b) Under Hi 



Figure 1: In Panel (a) we observe a realization under the null hypothesis (n = 
100). In Panel (b) we observe a realization under the alternative hypothesis 
(n = 100, ni = 40). 



which here means that a curve was used to simulate a fraction of the seg- 
ments by randomly choosing segments that are tangent to the curve at their 
midpoint (panel (b)). 

From a Statistics viewpoint, this detection task, that human subjects are 
asked to perform, can be formalized into a hypothesis testing problem. 

We say that a curve 7 C [0, 1]^, parametrized by arclength, interpolates 
{z,w) G [0, 1]^ X §^ if there is x such that 7(2;) = z and 7(x) = w, where 
denotes the unit circle and 7(x) the derivative of 7 at x. 

We observe n segments of fixed length dispersed in the unit square. 

• Under the null hypothesis, the segments have locations and orientations 
sampled uniformly at random in [0, 1]^ x S^. 

• Under the ( composite ) alternative hypothesis, the segments are as above 
except for ni of them that are chosen among those that a fixed curve 7 
interpolates. The curve 7 is unknown but restricted to belong to some 
known class F. 

Note that we do not specify the distribution of the segments tangent to the 
curve. 
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For 7 G r, define 

^nil) = = 1, . . . , n : 7 interpolates (Zj, Wi)}, 
and, witli some abuse of notation, 

ivr(r) = maxiy7(7). 

In 12], tfie test tliat rejects for large A^^(r) was analyzed for F the class 
of curves in the unit square with length and curvature bounded by some 
constant c > 0. In particular, it was shown that, under the null hypothesis, 
for some constants < A < B < oo, 

P{A n^/^ < A/'7(F) < 5 n^/^} ^ 1, n oo. 

Note that the upper bound implies that this test is powerful when rii > 

In this paper, we generalize this setting to higher dimensions. Let Q{k, d) 
be the set of fc-dimensional linear subspaces in W^. To G(A;, d) we associate 
its uniform measure A, which is the only invariant probability measure on 
G(A;, d) that is invariant under the action of the orthogonal group 0{d) - see 
[H], Section 1. 

For a function / : [0, 1]^ — > [0, l]'^ different iable at x, let 

f{x) = span {dsf{x) : s = 1, . . . , k} . 

A function / : [0,1]^ [0,1]'^ is said to interpolate {z,w) e [0,1^ x 
G{k, d) if there exists x G [0, l]'^ such that /(x) = z and f{x) = w. 

We consider the following hypothesis testing problem. We observe 
{{Zi, Wi) : I = 1,..., n} C[0, if X G{k, d). 

• Under the null hypothesis, {{Zi,Wi) : i = l,...,n} are independent 
and identically uniformly distributed in [0, 1]'^ x G{k, d). 

• Under the (composite) alternative hypothesis, {{Zi, Wi) : i = 1, . . . ,n} 

above except for ni of them that are chosen among those that a 
fixed function / interpolates. The function / is unknown but restricted 
to belong to some known class JF. 
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Before specifying JF, we introduce some notation. For a vector x = 
(xi, . . . ,Xd) € IR'^, the supnorm is defined as ||a;||oo = niax{|xj| : i = 1, . . . ,d}. 
For a function / : C M'^ M"^, ||/||oo = sup^gf^ ||/(a;)||oo. The Euclidean in- 
ner product and the corresponding norm are denoted by (■, ■) and || ■ || respec- 
tively. The angle Z {H, K) e [0, tt] between two linear subspaces H,K C. W^, 
with 1 < dimiJ < dimi^', is defined by 

- — ^ — - 

\\u\\ \\v\\ 

This corresponds to the largest canonical angle as defined in |3] and con- 
stitutes a metric on G(A;, d) - see also [T] for a related study of the largest 
canonical angle between two subspaces uniformly distributed in G{k,d). 

The class JF, parametrized hj [3 > 1, is defined as the set of twice differen- 
tiable, one-to-one functions / : [0, l]'^ [0, 1]*^ with the following additional 
properties: 

• For all s = 1/(3 < ||<9J(a;) ||oo < P for all x E [0,1]^ 

• For all s,t = 1, . . . ,k, \\dstf\\oo < (3; 

• For all s = 1, . . . , and x G [0, l]'^, 

Z {dsf{x), span {dtf{x) : t ^ s}) > 

which is void if = 1. (In this paper, we identify a non-zero vector 
with the one dimensional linear subspace it generates.) 

The last condition and the constraint (3 > 1 ensure that JF contains graphs 
of the form x — >■ {x,g{x)), where g : [0, 1]*^ [0, l]''"^ satisfies the first two 
conditions - see Lemma 16.11 
Define 

N7{f) = # = 1, . . . , n : / interpolates (Z„ Wi)], 
and, with some abuse of notation, 

iVr(^) = ma^ iVr(/). 

Let 

k 



k+{d-k){k + 2) 
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Theorem 1.1 There is a constant B = B {k , d, (3) < oo such that, under the 
null hypothesis, 

P {N;^{J^) > B nf^} -^0, n ^ oo. 

As before, this imphes that the test that rejects for large values of N^{J-') is 
powerful when ni > Bn''. 

Theorem 1.2 There is a constant A = A{k,d,l3) > such that, under the 
null hypothesis, 

P {N^{J^) < A nP} ^0, n ^ oo. 

The remaining of the paper is organized as follows. In Section 2 we 
introduce a related, yet different hypothesis testing problem. In Section 3 and 
Section 4, we prove results announced in Section 2. In Section 5 and Section 
6, we follow the arguments in Section 3 and Section 4 to prove Theorem ll.il 
and Theorem 11.21 Some intermediary lemmas are proved in the Appendix. 

2 Another Hypotheses Testing Problem 

We introduce another hypothesis testing problem as a stepping stone towards 
proving Theorem 11.11 and Theorem II. 2| and also for its own sake. 

Let a > 1, (3 > and define r = [aj = max{m G N : m < a}. (In 
this paper, we include in N.) Define the Holder class H'^''^(a,/3) to be the 
set of functions / : [0, 1]^ [0, l]'^, with / = (/i, . . . , /d) such that, for all 
s = (si, . . . , Sjfc) G with |s| = Si + ■ ■ ■ + Sfc < r, 

ll/^^l|oo</3; 

and, for all s G N'^ with |s| = r, 

\\f^'\x)-f^^\y)\U<P\\x-yr^-\ 

where f^^^ = {ds^.-.s^fi, ■ ■ ■ ^dsj^- s^fd)- When there is no possible confusion, 
we use the notation Ti. = H^''^~^(a, 
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Fix ro an integer such that 1 < Tq < r. Let 



S = {s G : |s| < ro}, 



with cardinahty 




s + k -1 
k-1 



) 



We denote by a vector in R*^'^ ^^'^l and by f^^\x) the vector (/'•^-'(x) : 
s e S). A function f e H is said to interpolate {x,y^) G [0, 1]'' x M('='-'^)|s| if 



• Under the null hypothesis, {{Xi,Y^^) : i = l,...,n} are independent 
and identically uniformly distributed in [0, l]''x [0, l]'^~''x [— /5, . 

• Under the (composite) alternative hypothesis, {{Xi,Yf) : i = 1, . . . ,n} 
are as above except for ni of them that are chosen among those that a 
fixed function / interpolates. The function / is unknown but restricted 
to belong to H. 

Figure El shows an example, with d = 2 and ro = 1. 



Consider the following hypothesis testing problem. We observe 



: i = C [0,1]'= X 



Define 



n : / interpolates {Xi,Y^^)}, 



and, with some abuse of notation. 



iv^)(7^) 



maxA^^''") 



Let 



k 1 



k + a{d — k)w l + a{d/k — l)w 



where 
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(a) Under Hq 
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Figure 2: In Panel (a) we observe a realization under the null hypothesis in = 
100). In Panel (b) we observe a realization under the alternative hypothesis 
(n = 100, ni = 40). 

Theorem 2.1 There is a constant B = B{k, d, a, (3, tq) > such that, under 
the null hypothesis, 



P{Ni'''\n) > B nf^'°^} ^0, 



n — i> oo. 



As before, this implies that the test that rejects for large values of Nn'^\l-i) 
is powerful when ni > BnP^^°\ 

Theorem 2.2 There is a constant A = A{k, d, a, j3, tq) > such that, under 
the null hypothesis. 



P{Ni''°\n) < A ^0, 



n — i> oo. 



Remark. For a = 2 and tq = 1, p(ro) = p, meaning that N^^°\n) and 
N^{T) are, in that case, of same order of magnitude with high probability. 
This will be used explicitly in Section 6 when proving Theorem 11.21 



3 Proof of Theorem EH 

For yf,y2 G M('^~'^)I^I, define the discrepancy 



«l>(yf,yf)=max||i/^-y2i^/(°-N). 



The discrepancy $ induces a discrepancy on functions, namely 
<|.(/,^?) = max||/(^)-^?(^)||:/("-N). 

The argument for proving Theorem 12.11 is based on coverings of 7i with 
respect to $. Let be the e-covering number of Ti with respect to $. 

Lemma 3.1 There is a constant Ci = Ci(fc, a, /5, rg) > such that, for all 
e>0, 

Lemma ITTl follows immediately from the proof of Theorem XIII in |5J, Chap- 
ter "e-entropy and e-capacity of sets in functional spaces" . 

For a set K C X M('^-'=)|s| and e > 0, we denote by K^'^ the set of 
points {x,y^) such that there is {xi,yf) G K with xi = x and ^{yf,y^) < e. 

For each e, we select an £-net {fj : j = 1, . . . , Lgr} of TC for $. For 
j = 1, . . . , Le, we define 

Kj = {(x,/) G M'^ X m('^-'=)|s| : <|.(/,/f ^(x)) < e} 
= graph«(/,■)^'^ 

where, for / G 7i, 

graph^(/) = {(x, f^\x)) : X G [0, l]'^} C [0, 1]'^ x m('^-'=)|s|. 

We extend N^''°\-) to subsets K G x M('^-'=)|s|, by defining N^^'^^K) 
to be the number of points {Xi, Yf) that belong to K. 
Let 

M„(e) = max Ni''°\Kj). 

j = l,...,Le 

By definition, it is straightforward to see that 

iV^)(7^) < M„(£), 
for all e > 0. We therefore focus on bounding Mn{e). 

By Boole's inequality, we have 

P {Mn{e) >b}<L,- max P {Nj[''\Kj) > b} . 

j = l,...,Le 
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Moreover, we know that, for any set C M'^ x R^*^ ^^1^', 

iV^)(ir)^Bin(n,/i(ir)), 

where /i is the uniform measure on [0, l]'^ x [0, l]'^"''' x [— /5, 
Hence, 

P{Mn{e)>b} < L,- max P {Bm{n, fi{Kj)) > b} 

= ■ P |Bin(r;., max li{Kj)) > fej . 

Lemma 3.2 There is C2 > such that, for all f ^Ti and all e > 0, 

/x(graph^(/)^'*) < C2£('^^^)"'. 

Proof of Lemma Vj.^/^ Start with 

graph«(/)^'* C U,,e[o,i]. {x} (g) B(/(^)(x), e^-l^l/"), 

ses 

where B(?/, r/) denotes the supnorm ball in centered at y of radius rj. 
Hence, integrating over x G [0, 1]*^ last, we have 

/x(graph^(/)^'*) < csJJ (5^-1^1/")'^-'= = C2 e^''-^^'^. 

SGS 

□ 

Using Lemma f3. 21 we arrive at 

P {M„(e) >h}<Le-P {Bin(n, cae^'^"^)"') > h] . 

Lemma 3.3 There is a constant c > such that, for any n positive integer 
and p G (0, 1/2), and for all b > 2np, 

P {Bin(n, p) > b} < exp(— c ■ b). 

Lemma 13.31 follows directly from standard large deviations bounds for bino- 
mial random variables - see [S], p. 440, Inequality 1, (ii). 
We use Lemma IX^ to obtain 

P {Bin(n, cse^'^"'^)"') > b} < exp(-c ■ b), 
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for all b > 2nc2£(^-^)"'. 

Collecting terms, we arrive at the following inequality, valid for B > 2c2, 

P{Mn{e)> B < exp (de"^/" - C3 B e'^'^-'^^'^n) . 

Choose e = so that e'^^/" = e('^-'=)"'n = nP^'^'l Then, the 

result above transforms into 

P {Mn{e) > B nP^'°^} ^0, n ^ 00, 

valid for B > (ci/cs) V 2c2. 

4 Proof of Theorem 12.2 

We use the notations appearing in Sectional except for the various constants 
which are refreshed in this section. 

For each s G S, take ^/'g ^ IK'^ infinitely differentiable, supported in 
[-1/2, l/2]^ and satisfying ^lji^\o) = if t G S and t s; and i/ji^\o) = 1. 

Let ci > 1 such that ci > H^/'s*^ ||oo, for all s e S and t G N'' with 
|t| < r + 1. 

Again, choose e > such that e^''^" = e^'^^^^'^n. Define Ss = er^^l^l/" 
and, with C2 > 1 to be determined later, let e' = {c2£y^"- We partition 
[0, 1]'^ into hypercubes of sidelength e', that we denote by J^, where m = 
(mi, . . . , mfc) G ([0, 1/e'] fl N)'^. For a given m, let 

Rm = ImX [e/2, e]'^-'' x JJ [0, e^^''' cR'' x m('='"^')|s|_ 

ses\{o} 

Denote by = ([0, l/e'] H 2N)'' and let M be the (random) subset of 
m E Ai such that there is z(m) G {1, . . . ,n} with (Xj(m), y^m)) ^ 

Lemma 4.1 There is a constant A = A{k,d,a, f3,rQ) > such that 

P{|M| > A n^(^«)} ^ 1, n^oo. 

Lemma f4. II is proved in Appendix^ Theorem 12.21 now follows if we are able 
to interpolate the points {{X^m), ^jfm)) • ™ ^ ^} "with a function in TC. 
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Define, for each j — 1, . . . ,d — k, 



where 
with 



ses 

Note that, for t G N^ 

o<t'<t ^ ^ ses 
where if s = (si, . . . ,Sk) and t = {ti, . . . ,tk), 

s / V *i / V 

Hence, g'j%{0) = Y^^^^^.. This imphes that hfl^{Xi^^)) = Y.^^^^^., and so 

/ij^''(Xj(^)) = ^j(^) J-, since for each j = 1, . . . ,d — k, the /ij,m's have non- 
overlapping supports. Therefore, if we let /i = (/ii, . . . , hd-k), we have that /i 
interpolates each point in {(Xj(^), Y^^^-^) : m e M}. 

Remains to prove that, for each j — 1, . . . ,d — k, hj e H'^'^(q;, Fix 
j — 1, . . . ,d — k. Again, because for every x e [0, 1]*^ there is at most one m 
such that hj^jn{x) 0) we have for all t e N*^, 



Fix m e M. We have ||M*i||oo - hflWoo (s')"'*', with 



o<t'<t ^ ^ ses »Mj 



Since I^y^n . < e and 
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we have, for |t| < r + 1, Hsfj^mlloo < C3 Cj''" e, with C3 = 03(0;, A;) > 0. In 
particular, C3 does not depend on the choice of C2. Choose C2 > 1 such that 

C3 4/"-' < p. 

Hence, for all t such that |t| < r + 1, 

||'^j,TOl|oo S C3 C2 e j ' ' - C3 C2 e ' " . 
This implies that, for e small enough, hj takes values in [0, 1] and 

\\hf\\oo<^, 

for all t such that |t| < r. 

Remains to prove that, for all t such that |t| = r, 

\hf{x')-hf{x)\<p\\x'-xr^\ 

for all G [0, 1]''. 

• Suppose — x||oo > 

\hf{x')-hf{x)\ < \\hf\U 

< ccl^'^ eie'nWx-x'lUe'r^^ 

^ r/a—l II /||Q,_j. 

< C3 \\x-x • 

• Suppose lis — s'lloo < and let t+ = (ti + 1, . . . 

\hf{x')-hf{x)\ < \\h^;^\U\\x-x'\U 



r/cj — 1 II /||q:— r 

— C3 C2 11^ ^ ^ II 00 • 



Since we chose C2 such that C3 Cg''" ^ < /5, we are finished. 

5 Proof of Theorem 11.11 

Let \l/ be the discrepancy on M*^ x G(A;, c?) defined by 

^((z, if), (^1, = max{||z - |U, ^ i^i)'}- 



I oo 
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Each / G is identified with (/, /), viewed as a function on [0, 1]^ with 
values in M'^ x G[k, d) - the first and third constraints on the derivatives of 
/ G guarantee that f{x) is indeed a fc-dimensional subspace of for all 
X G [0, 1]*^. With this perspective, ^ induces a discrepancy on JF. The proof 
is based on coverings of with respect to that discrepancy - still denoted 
by 

It turns out that \E' is dominated by the discrepancy $ defined in Section 
EJ with a = 2 and ro = 1. Indeed, we have the following. 

Lemma 5.1 There is a constant c = c{k,d,f3) such that, for any f,g & J-" 
and X G [0, 1]'', 

^(f{x),g{x))<c max \\dj{x) - dsg{x)\\^. 

\ / s=l,...,k 

To get Lemma l5.ll we apply Lemma ID. II in Appendix lUl with Ui (resp. Vi) 

defined as dif{x) (resp. dig{x)) and Ci = 1//5, = P, = l/{2[3{d — k)). 

Therefore, the e-covering number of with respect to \E' is bounded by 
the e-covering number of T with respect to $, whose logarithm is of order 
_ Lemma f3.H where d enters only in the constant. 

Following the steps in Section |21 we only need to find an equivalent 
of Lemma 13.21 namely compute an upper bound on the measure of the e- 
neighborhood of 

{(/(x),/(x)):xG[0,l]'=}cM'^xG(fc,rf) 

for the discrepancy \E', valid for all f E J^. For H G G(A;, d), let 

B{H, e) = {K G{k, d) : Z {H, K) < e}. 

As in the proof of Lemma f3. 21 we are left with computing a upper bound on 
X{B{H,e)), which is independent of G G{k,d) since A is invariant under 
the (transitive) action of the orthogonal group. (Remember that A denotes 
the uniform measure on G(fc, d).) 

Lemma 5.2 There is a constant c = c{k, d) such that, for all e > and for 
all H G G{k, d), XiB{H, e)) < c e^'^-^)'' . 

Lemma 15.21 is a direct consequence of Lemma IB. 11 in Appendix IHl and the 
fact that A {Q{k,d)) = 1. 
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6 Proof of Theorem 11.21 



We show that Theorem 12.21 imphes Theorem I1.21 We first start by showing 
that functions of the form /(x) = {x,g{x)), with g belong to JF. 

Lemma 6.1 For all g ETi, the function f{x) = {x,g{x)) belongs to T . 

Lemma f6. II is proved in Appendix O 

Let W be sampled uniformly at random in G(A;, rf). With probabil- 
ity one, there is a unique set of vectors in M'^"'^, : |s| = 1}, such 
that W = span{(s, y^) : |s| = 1}. Indeed, W has the same distribution 
as span {wi, . . . , Wk}, where Wi, . . . ,Wk are i.i.d. uniformly distributed on 
the unit sphere of M'^ and therefore, with probability one, {wi, Ci) ^ for all 
z = 1, . . . , fc, {cj : i = 1, . . . , ci} being the canonical basis of M'^. The unique- 
ness comes from the fact that a subspace of the form span {(s, F®) : |s| = 1} 
does not contain a vector of the form (0, Y\ with Y G R'^"'^ \ {0}, so it does 
not contain two distinct vectors of the form (s, Yi) and (s, 1^2)- 

Through the map k that associates W to : |s| = 1}, the uniform 
measure on d) induces a probability measure v on ]R('^^^)'^. If we observe 
{(Zj, IVj) : i = 1, . . . ,n}, we let 

Zi = {Xi, Yi) eR'^x M.'^-^ and Wi = span {(s, Y^) : |s| = 1} . 

With here S = {s G N'^ : |s| < 1}, we thus obtain {(X^, F-^) : i = 1, . . . , n}, 
independent and with common distribution A^^A^.^®// = A^®//, where A^ 
is the uniform measure on [0, 1]^. Note that, if g interpolates {{Xi, Y^^) : 
i = 1, . . . ,n}, then / defined by /(x) = (x, g{x)) belongs to by Lemma lUTTl 
and interpolates {{Zi,Wi) : i = 1, . . . ,n}. With A^ z/ playing the role of 
fx, the uniform measure on [0, l]'^ x [0, 1]'^"'^ x [— /3, the present 
setting parallels the situation in Section 4. Following the arguments given 
there, we are only left with obtaining the equivalent of Lemma f4. II Looking 
at the proof of Lemma 14.11 in Section \^ all we need is a lower bound of the 
form 



= c e 



k/2+{d-k){l+k/2) 



(Here a = 2 and w = 1 + k/2.) Because 



Ad ® i^{Ro) = ce 



l/2](d-fc)fc 



the following lemma provides what we need. 
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Lemma 6.2 There is a constant c = c{k, d) > such that, for all e > 
small enough, 

To prove Lemma 16.21 we first show that for some constant c = c{k, d) > 0, 
K~^{[0, contains B{H, ce), where H = span {ei, . . . , 6^} and {ei, . . . , e^} 

is the canonical basis of M'^. Indeed, let c be the constant provided by 
Lemma [0.21 and consider K G B{H,ce). Let n{K) = {y^, . . . ,y'^}, so that 
K = span {cj + : z = 1, . . . , k}. Applying Lemma ID. 21 with Ui = Ci for 
i = 1, . . . , k, V = Cj + and Uk+i = {v — Pv)/\\v — Pv\\qo, where P denotes 
the orthogonal projection onto H, we get that Z (t>, H) > c \\v — Pv\\oo- With 
||f — Pf lloo = ||z/"'||oo and the fact that Z {v, H) < Z {K, H), we see that we 
have lly-'lloo < £■ This being true for all j, we have fi{K) G [0, e]''^'^~''\ We 
then apply the following result. 

Lemma 6.3 There is a constant c = c{k, d) > such that, for all e > and 
for all H G G{k, d), \{B{H, ce)) > c 

Lemma 16.31 is a direct consequence of Lemma IB. 21 in Appendix ^ and again 
the fact that A {G{k, d)) = 1. 

A Proof of Lemma 14.11 

Lemma [4. II is a conditional version of The Coupon Collector's Problem - see 
e.g. j7j. We nevertheless provide here an elementary proof. 

Let K = Nn°\Ura&M Rm) ■ We kuow that K ~ Bin(n,p), where p = 
\M\ fi{Ro) with \M \ oc and 

fiiRo) = {ef ■ {8/2)"-^ ■ n {e\s\imY-'' oc {e')h^'-^^^. 

ses\{o} 

This implies that pn = c'\Ai\ for some c' > not depending on n, by 
definition of e and e'. Let c = c'/2 and Cq G (e~^, 1), and also, to simplify 
notation, let i = \M\ and S = \M\ — \M\. Because \M\ oc n^^**"), it is enough 
to show that P {S* > cq ^} — > as — > cxd. 
We have 

P {S > Co £} < P {S > Co i\K > c i} + P {K > c i} , 
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with P {K > c ^} — s> as n ^ oo by Lemma f3 .31 and 

P{S>co£\K>c£}<P{S> Co i\K = \c £]} . 
Using Chebychev's inequality, we get 

vSir{S\K = \ce]} 



P{S> Co i\K= \ci]}< 



{coe-E{S\K=\ci]})''- 
We know that for any non-negative integer k, 

E{S\K = k} = i{l-l/i)\ 

and 

var{^|ir = A;} =£((l-l/£)^-(l-l/£)2^)+£(£-l)((l-2/^)^-(l-l/£)2^). 
Therefore, when i —>■ oo, 

E{S\K= \c £]} 

and, for all i, 

vav{S\K= \ci]}< cii, 

so that, when i is large, 

var{S\K= \c i]} ^ 

(co £-E{5|ir = \ce\}f - i ' 

Since i is an increasing function of n that tends to infinity, we conclude that 

P{S>Coi}^0, n^oo. 

□ 



B Coverings of G(/c, d) 

Lemma B.l There is a constant c = c{k,d) > such that, for all e > 0, 
there zs Hi, . . . , e G{k,d) with (. > c e-'^'^'''^^ and B{Hi,e)nB{Hj,e) = 
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Proof of Lemma \B.1\ Fix e > and consider 
Hn = span Ici + e n-ijCj 

I j=k+l 



: i = 1, . . . ,k 



where n = (n^j : i = l,...,k;j = k + l,...,d) G ([0, 1/e] n N)^'^-*^)*^. As 
there are more than 1/2 s-i<^'^)^ such -ffn's, it suffices to prove that, for some 
constant c > 0, 

Z (if^, > c £, 

as soon as m 7^ n, for that would imply that the balls B{Hn,c/3 e) are 
disjoint when n runs through ([0, 1/e] fl N)'^'^"'^^'^. 

Therefore, fix m ^ n, both in ([0, 1/e] n N)^'^-'')*^. For z = 1, . . . , fc, let 

Ui = + £j2'j=k+i^'i',j^j Vi = Ci + ^Yl'^j=k+i^i,i^v where we assume, 
without loss of generality, that Vi. Now, by definition 

Z {H^, Hn) > Z span {ui, Uk}) . 

To proceed further, we apply Lemma fD.2l with ui,...,Uk, Uk+i = (fi — 
Ui)/\\vi — Milloo and v = Vi. It is straightforward to see that the conditions 
are satisfied, since in particular vi = ui + \\vi — ui\\ooUk+i. Hence, for a 
constant c > depending only on k, d, 

Z (t;i,span{Mi, . . .,Uk}) > c \\vi - Ui\^. 

To conclude, note that ||fi — mi||oo > ^- D 

Lemma B.2 There is a constant c = c{k,d) > such that, for all e > 0, 
there is Hi,..., He e G{k,d) with £ <c and G{k,d) C [j-B{Hi,e). 

Proof of Lemma \B. M Let ei, . . . , be the canonical basis for M*^ and let ci 
be the constant given by Lemma ID .41 As in the proof of Lemma IB.H define 



H^ = span <^ e^(i) + e ^ ^ij^^d) -i = l,---,k 
I j=k+i 

where n = {riij : i = 1, . . . , k; j = k + 1, . . . , d) e ([0, (ci + l)/e) n N)('^-'=)^ 
and cr is a permutation of {1, ... , d}. There are no more than c s~i'^-f')'' such 
subsets, where 

d\ ^c^^iYd-k)k_ 



k 
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We now show that there is a constant c = c{k,d) > such that, for all 
H C G{k, d), there exists such an satisfying Z {H^, H) < c e. 

So fix if C G(fc, d). From Lemma fP. 41 it comes that there is a, a permu- 
tation of {1, ... , d}, and £ [0, ci] for z = 1, . . . , and j = /c + 1, . . . , d, 
such that 

H = span <^ e^(i) + ^ 6,<70)ea(i) : i = 1, . . . , > . 

For z = 1, . . . ,k and j = k + 1, . . . ,d, define riij to be the entire part of 
6,f7(i)/^; thus obtaining n = (riij) G ([0, (ci + !)/£:) fl f^Y'^-''^''. Applying 
Lemma [p. II with, for i = 1, . . . , k, 

d 

j=k+l 

and 

j=k+i 

we get Z (-f^n) H) < c e for some constant c = c(A;, (i) > 0. □ 



C Proof of Lemma 16.11 

Only the last property defining is non-trivial. Fix / G 7i, and let /(x) = 
{x,g{x)). Fix s G {1, . . . , fc} and x G [0, 1]*^, and let s be the s^^ canonical 
basis vector of M'^. We have dsf{x) = (s, dsg{x)). 

Let V = V1+V2 where vi = (s, 0) and V2 = (0, dsg{x)), and pick one vector 
w G span {(9t/(a;) : t 7^ s}. 

We first show that 

/ \2 ^ (d — 11 II ||2 

M < i + (,_,)^. H ■ 11-11- 

Indeed, since fi is orthogonal to w, we have {v,w) = {v2,w), so that, using 
the Cauchy-Schwartz inequality. 



{v, w)'^ < 11^211^ ■ \\w 



|2 
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We have WvW^ = + \\v2\\'^- Moreover, ||fi||^ = 1 and 11^211^ < {d — k)f3'^, 

since ||9s/(x)||oo < P- Conclude with 

Since w is arbitrary in span {dtf{x) :ty^s}, this shows that 
Z {dsf{x), span {dtf{x) : t ^ s}) > acos 

Furthermore, 

/ / id-k)(3^ \ ( 1 \ 

acos 4 / — — — ■ = atan — , 

\^l^{d-k)(5'^) \p,/d^J 

1 

> ; , 

~ 2(3^1^ 

where the last inequality comes from the fact that atan(?/) > y/2 for y <ti /2. 



{d - k)(3^ 
l + {d- k)f3^ 



D Auxiliary Results in Euclidean Spaces 

Lemma D.l Let ci, C2, C3 be three positive constants. Let ui, . . . , Uk', vi, . . . ,Vk G 
such that for all i = 1, . . . , k, ci < \\vi\\oo, ||Mi||oo < C2, and, if k > 2, 

Z {ui, span {uj : j ^ i}) > C3; 

Z {vi,span{vj : j ^ i}) > C3. 
Then, for a constant c depending only on k, d, ci, C2, C3, 

Z (span {uj : i = 1, . . . , A;} , span {vi : i = 1, . . . , k}) < c max \\vi — Uj||oo. 

i=l,...,fe 

Proof of Lemma W. 11 By multiplying the constants that appear in the Lemma 
by constants that depend only on d, we can assume that the conditions in the 
Lemma hold for the Euclidean norm. Throughout, let e = maxj=i_ ^ \\vi — 

II oo- 
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First assume that Ui, . . . ,Uk (resp. Vi, . . . , Vk) are orthonormal. Take 

iiUi e span {wj : i = 1, . . . , A;} , 



u 



of norm equal to 1. Define 

= ^ ^jtij e span {tij : i = 1, . . . , A;} . 

i 

We show that 

acos(|(ii, ^;)|) = 0{e), 

by showing that 

{U,V) = l + 0(£'). 

This comes from the fact that, since ||rt|| = ||f|| = 1, 

{u,v) = 1 - \\u-v\\^/2, 

and 

\\u — v\\ < ^ l^il \\ui — Vi\\ < y/d e. 

i 

If Ml, . . . ,Uk (resp. Vi, . . . ,Vk) are not orthonormal, we make them so. 
Define a[ = Ui and Oi = a'l/ll'^'ill' for i = 2, . . . , k, define 



•i-i 



a[ = Ui - > ^■"i) 



and ai = a'J\\a[\\. Similarly, define b[ = vi and bi = b[/\\b[\\, and for 
i = 2, . . . , k, define 



i-l 



bl^Vi-^{vi,bj)bj, 



and k = by\m. 



We have, for i = 1, . . . ,k, Cisinca < ||a^|| < C2. Indeed, since a'^ is the 
difference between Ui and its orthogonal projection onto span {ui, . . . , Wj-i}, 
it follows that 

||a-|| = \\ui\\ sin Z {ui, span . . . , Ui_i}) , 



20 



with 

Z {ui, span . . . , Ui_i}) > Z (wj, span {uj : j ^ i}) > C3 > 0. 

In the same way, for i = 1, . . . , /c, ci sin C3 < < C2. 

We also have a[ — b[ = 0{e). We prove that recursively. First, — = 
\\ui — < £. Assume a^_]^ — h[_]^ = 0{e). This implies aj_i — bi-i = 0{e); 
indeed, 

— Oj-i — — 



1*^4-1 II iri-1 1 



< (ll«UII + Q(g)K -i-||<-illK-i + Q(g)) 



C2 



= 0(s). 

Now, 

a- - 6- = Mi - - ai_i)ai_i - {vi, 6i_i)6i_i), 
with — = 0{s) and 

(vi, = {vi, ai_i + 0(£))(aj_i + 0{e)) = aj_i)aj_i + 0{e). 

So that 

(tij, aj_i)aj_i - {vi, = (wj - Vi, aj_i)ai_i + 0(£) = 0{e). 

Hence, the recursion is satisfied. 

We then apply the first part to ai, . . . , and 61, . . . , 6^. □ 

Lemma D.2 Fix Ci, 02,03 three positive constants. Let Ui, . . . ,U]^+i e M."^ 
such that for i — 1, . . . , k + 1, Ci < \\ui\\oo < C2, and 

Z {ui, span {uj : j 7^ z}) > C3. 

r/ien, t/iere zs a positive constant c depending only on k, d, Ci, C2, C3 suc/i t/iat, 
/or a// V = Yji^i'^i '^^th \\v\\oo < C2, 

Z{v,spein{ui : i ^ 1, . . . ,k}) > c (|Cfe+i| A 1)- 
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Proof of Lemma W.Sl The proof is similar to that of Lemma ID. II above. 
Again, we may work with the Euchdean norm instead of the supnorm. 

First assume that ui, . . . , Uk+i are orthonormaL Take v = ^iUi of norm 
equal to — '^2- Let Pv = ^j^i^i^i, the orthonormal projection of v 

onto span {ui, . . . ,Uk}- By definition, 



Z (f , span {ui, . . . , Uk}) = acos 
We then conclude with 



{v,Pv) 
\v\\ \\Pv\\ 



In general, we first make ui, . . . , Uk+i orthonormal as we did in the proof 
of Lemma [P.ll except in reverse order, meaning that ak+i = Uk+i/\\uk+i\\- 
Since for all v = Y^i^i^i = Y^iliai, \lk+i\ = \^k+i\\\uk+i\\ > ci we can 

apply the first part to Oi, . . . , a^+i. □ 

Lemma D.3 Let Ci, . . . ,6^ be the canonical basis ofM.'^. There is a constant 
c = c{k, d) > such that, for every Ui, . . . , Uk, orthonormal set of vectors in 
W^, there exists a permutation a o/ {1, . . . , c?} such that 

KMi,e^(i))| > c, Vi = l,...,/c. 

Proof of Lemma \D. 31 We prove Lemma lD.31 bv recursion on k. For /c = 1, 
we may choose c(l,(i) = l/\fd, since Ui is of norm 1. Suppose the result is 
true at A; — 1, and consider the case at k. Without loss of generality, we may 
assume that 

\{ui,ei)\ >c{k-l,d), Wi = 1, . . . ,k - 1. 
We need to show that there is a constant Ci > and j G {k, . . . ,d} such that 

\{uk,ej) \ > ci. 



As Lemma ID. 31 implies Lemma ID. 41 we can use the latter at — 1 to get 
vectors Ci + Vi, i = 1, . . . , k — 1, with Vi G span {e^, . . . , e^} , and \\vi\\oo < C2, 
such that 

span {d +vi,..., Ck-i + Vk-i} = span {ui, . . . , Uk-i} . 
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Let ^ = maxj=fc^,,^rf | {uk-, ej)\. For alH = 1, . . . , A; — 1, we have 

\{uk,Vi) \ <{d- k)c2^. 
Now, since for alH = 1, . . . , A; — 1, (ufc, + Vi) = 0, we also have 

\{uk,ei) \ <{d- k)c2^. 

Since there is z = 1, . . . , c? such that | {uk, e.i) \ > 1/Vd, we must have 

1 1 
^ > ^ V . 

\'d y d{d — k)c2 

Conclude by calhng the right handside C3 and letting 

c{k, d) = c{k — 1, (i) A C3. 

□ 

Lemma D.4 Let ei, . . . , he the canonical basis ofM.'^. There is a constant 
c = c{k, d) > such that, for ui, . . . ,Uk any orthonormal set of vectors in 
W^, there exists a permutation o 0/ {1, . . . , ci}, such that 

span {ui, . . . , Ufc} = span {e<^(i) + . . . , e<^(fc) + 

where, for all i = 1, . . . , k, 

Vi G span {e„(j) : j = k + 1, . . . , d} , 

and ||fi||oo < c. 

Proof of Lemma \D.4\ Applying Lemma ID. 31 there is Ci > and a per- 
mutation a such that 

|(Mi,e<x{i))| > ci, \fi = l,...,k. 

Without loss of generality, suppose a = id. 

We now triangulate the matrix with column vectors ui, . . . ,Uk- In other 
words, we consider {u[ : i = 1, . . . , k}, where u[ is the orthogonal projection 
of Ui onto span {cj, e^+i, ■ ■ ■ , e^}. For alH = 1, . . . , /c, we have = ^jCj + Wi, 
where > Ci and Wi G span {e^+i, . . . , e^} with ||wi||oo < 1- Define Vi = 
Wi/^i and conclude with the fact that 

span {u'l, . . . , %} = span {ui, . . . , Uk} ■ 

□ 
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